Modeling pronunciation, rhythm, and intonation for automatic assessment of speech quality in aphasia rehabilitation

Patients with aphasia often have impaired speech-language production skills, resulting in tremendous difficulties in tasks that require verbal communication. To facilitate rehabilitation outside of therapy, we are collaborating with the University of Michigan Aphasia Program (UMAP) to develop an automated system capable of providing feedback regarding the patient’s verbal output. In this paper we introduce a robust method for extracting rhythm and intonation features from aphasic speech based on template matching. These features, combined with Goodness of Pronunciation (GOP) scores and our previous feature set, help our system achieve human-level performance in classifying the quality of speech produced by patients attending UMAP. The results presented in this work demonstrate the efficacy of our technique and the potential of this system for handling natural speech data recorded in non-ideal conditions as well as the unpredictability in aphasic speech patterns.